997 Association for Computational Linguistics Recurrent Neural-network Learning of Phonological Regularities in Turkish 1 Network Architecture

Simple recurrent networks were trained with sequences of phonemes from a corpus of Turkish words. The network's task was to predict the next phoneme. The aim of the study was to look at the representations developed within the hidden layer of the network in order to investigate the extent to which such networks can learn phonological regularities from such input. It was found that in the diierent networks, hidden units came to correspond to detectors for natural phonological classes such as vowels, consonants, voiced stops, and front and back vowels. The initial state of the networks contained no information of this type, nor were these classes explicit in the input. The networks were also able to encode information about the temporal distribution of these classes. The network used is a simple recurrent network of the type rst investigated by Elman (Elman, 1990). It consists of a feedforward network, supplemented with recurrent connections from the hidden layer. It was trained by the back-propagation learning algorithm (Rumelhart, Hinton and Williams, 1986). The ability of such networks to extract phonological structure is well established. For example, Gasser (Gasser, 1992) showed that a similar network could learn distributed representations for syllables when trained on words of an artiicial language. Figure 1 shows the architecture of the network. Within this network architecture, four diierent network conng-urations were investigated. These all had 28 units in both the input and output layers; they varied only in the number of units in the hidden layer, ranging from two to ve.